1. Field of the Invention
This invention relates to an image processing apparatus, image processing method, an image pickup apparatus, a computer program and a recording medium adapted to eliminate noises, while storing edges of images.
2. Description of the Related Art
When an image is picked up by a camera, the color tone of the image depends on lighting to a large extent. For example, the picked up image of a subject bears an orangey color tone when the subject is irradiated with candle light but bears a bluish color tone when it is irradiated with moon light. Images of a subject may be picked up with completely different color tones on the same site depending on lighting.
When picking up an image, it is possible to correctly reproduce the original color tone by utilizing natural light without using lighting equipment such as a flash bulb. However, the picked up image contains noises to a considerable extent when the subject is shot indoors or at night because of insufficient exposure to light. When, on the other hand, flash light is used for imaging, the edges of the objects and other details in the picked up image appear clearly but the color tone of the image may not be accurate and/or shades and high lights that do not actually exist may also appear.
An image processing apparatus as illustrated in FIG. 1 of the accompanying drawings has been proposed to dissolve the above-identified problem. Referring to FIG. 1, the image processing apparatus 11 includes a pair of low pass filters 12, 13, a high pass filter 14, a pair of image synthesizing sections 15, 17 and a single shadow extracting section 16.
The low pass filter 13 is a cross bilateral filter that detects edges from an image picked up by using flash light (to be referred to as flash image hereinafter) and remove noises from an image picked up without using flash light (to be referred to as natural light image hereinafter). The high pass filter 14 extracts edges of objects in the picked up image. For extracting edges, the image processing apparatus executes a process of dividing each pixel value in the natural light image by the corresponding one of the flash image.
The image synthesizing section 15 synthetically combines the natural light image from which noises are removed by the low pass filter 13 and the flash image from which edges are extracted by the high pass filter 14 to generate a synthetic image Comp. For synthesizing an image, the image processing apparatus executes a process of multiplying each pixel value of the low pass filter 13 by the corresponding one of the output image of the high pass filter 14. The synthetic image Comp provides the advantages of the flash image and those of the natural light image so that its color tone is accurate and it contains fewer noises.
The low pass filter 12 removes noises from the natural light image. A bilateral filter is used for the low pass filter 12. A bilateral filter can detect edges and remove noises from a single image.
The shadow extracting section 16 extracts differences between the two images including the flash image and the natural light image and evaluates the probability by which the flash image is changed from the natural light image by high lights and shadows. The image synthesizing section 17 carries out an operation of weighted addition of the output image from the low pass filter 12 and the output image Comp of the image synthesizing section 15 according to the outcome of the evaluation of the shadow extracting section 16. More specifically, the image synthesizing section 17 raises the weight of the image in a part thereof where shadows and high lights may highly probably have been produced by flash light and reduces the weight of the image in a part hereof where they may probably have not been produced. On the other hand, the image synthesizing section 17 removes unnecessary shadows and high lights from the synthetic image Comp and generates an output image Out.
As described above, the known image processing apparatus 11 can obtain an image where edges in a flash image and the color tone of a corresponding natural light image are optimally blended by generating two images including an image where edges are synthesized from a flash image and an image from which noises are removed without referring to the edges of the flash image, raising the coefficient of the output image of the high pass filter 12 for pixels where shadows and high lights may highly probably have been produced by flash light and reducing the coefficient of the output image Comp of the image synthesizing section 17 for pixels where shadows and high lights may probably have not been produced (see, for example, Non-Patent Document 1: Georg Petschnigg et al, Digital Photography with Flash and No-Flash Image pairs, acm Transaction on Graphics, Vol. 23, Number 3, pp. 664-672, August 2004).